Methods and apparatus for enhancing a digital content experience

ABSTRACT

Aspects of the present invention include methods and associated apparatuses that enhance a digital content experience. The methods and associated apparatuses may receive a selection of a video to view and determine a variety of content related to the selected video. The related content may be presented to a viewer while the video is playing.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage entry of International ApplicationNo. PCT/US2012/066839, filed Nov. 28, 2012, which claims priority toU.S. Patent Application No. 61/564,123, filed Nov. 28, 2011. Thedisclosures of the prior applications are hereby incorporated in theirentirety by reference.

BACKGROUND

Today's electronic devices, such as computers, tablets, smart phones,personal digital assistants (PDAs), etc., enable users to access and/ordownload various videos on-line. A user may access and/or download avideo from an on-line content provider to play on the user's electronicdevice. Typically, when a selected video is played, the video will bethe only content item being displayed to the user. If a user wants toperform other tasks and/or view other content on the electronic device,the user generally has to interrupt the video experience to engage withother online experiences.

Thus, it would be desirable to provide a mechanism that allows a videoto be played while presenting additional contextually relevant contentto the viewer of the video in a single, related experience. It would befurther desirable to provide mechanisms that can use elements of thevideo as a dynamic search query to present additional related content tothe video while the user is viewing the video. In addition, it wouldalso be desirable to provide mechanisms that can recommend videoselections and associated related content for users to view.

SUMMARY

The following presents a simplified summary of one or more aspects ofthe present invention in order to provide a basic understanding of suchaspects. This summary is not an extensive overview of all contemplatedaspects, and is intended to neither identify key or critical elements ofall aspects nor delineate the scope of any or all aspects. Its solepurpose is to present some concepts of one or more aspects in asimplified form as a prelude to the more detailed description that ispresented later.

Aspects of the present invention include methods and associatedapparatuses that, among other things, enhance a digital contentexperience, such as a video selection, recommendation and relatedcontent viewing experience. The methods and associated apparatuses mayrecommend a selection of a video for a user to view, for example, basedupon the user's preferences and/or browsing behaviors. The methods andassociated apparatuses may also receive a selection of a video to viewand determine a variety of related content to the selected video. Therelated content may be presented to the viewer while the video is beingplayed. Thus, a viewer may be able to view a video while accessingand/or interfacing with a variety of related content to the video on asingle screen, without having to stop the video and/or transition toanother interface to access or interact with the related content.

To the accomplishment of the foregoing and related ends, the one or moreaspects comprise the features hereinafter fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more aspects. These features are indicative, however, ofbut a few of the various ways in which the principles of various aspectsmay be employed.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements, andin which:

FIG. 1 is an illustration of a connectivity system in accordance with anaspect of the present invention;

FIG. 2 is an illustration of an example video transcript componentoperable within the connectivity system in accordance with anotheraspect of the present invention;

FIG. 3 is an illustration of an example related content engine operablewithin the connectivity system in accordance with another aspect of thepresent invention;

FIG. 4 is an illustration of an example content feed in accordance withan aspect of the present invention;

FIG. 5 is an illustration of an example method flow for creating a videotranscript in accordance with an aspect of the present invention;

FIG. 6 is an illustration of an example method flow for determiningrelated content to the video in accordance with an aspect of the presentinvention;

FIGS. 7A and 7B present illustrations of an example method flows forentity extractions and facet extractions in accordance with an aspect ofthe present invention;

FIG. 8 is an illustration of an example method flow for determiningrelated content for the video in accordance with an aspect of thepresent invention;

FIGS. 9A-9M present illustrations of example graphical user interface(GUI) screens illustrating exemplary implementations of aspects of thepresent invention;

FIG. 10 illustrates various features of an example computer system foruse in conjunction with aspects of the present invention; and

FIG. 11 illustrates an example system diagram of various hardwarecomponents and other features for use in accordance with aspects of thepresent invention.

DETAILED DESCRIPTION

Various aspects are now described with reference to the drawings. In thefollowing description, for purposes of explanation, numerous specificdetails are set forth in order to provide a thorough understanding ofone or more aspects. It may be evident, however, that such aspect(s) maybe practiced without these specific details.

Aspects of the present invention relate to methods and apparatuses forleveraging a video viewing experience, such as a video selection,recommendation and related content viewing experience. In one aspect,the methods and associated apparatuses use a selected video as aconsumer interest indicator to determine and/or recommend other contentthat a viewer of the video may also be interested in. For example, aviewer may signal interest in content by: reviewing and/or ratingcontent, providing comments on content, sharing content with otherusers, and selecting content to view and/or access. In an aspect, themethods and apparatuses may use a video transcript as a search query tosearch content databases to determine other related content to theselected video. The related content may be presented to a viewer whilethe video is being played. Thus, a viewer may be able to view a videowhile viewing and/or accessing a variety of related content, withouthaving to stop the video and/or transition to another interface toaccess the related content (e.g., the video and related content may bepresented contemporaneously within the same interface). Therefore, theapparatus and methods may provide a video viewing experience thatpromotes content multitasking while viewing a video.

The methods and associated apparatuses may also allow viewers tonavigate to specific time markers in the video from the related content.For example, when viewers access an item of related content associatedwith a time marker in the video, the video may move to the time markerassociated with the accessed related content. Thus, viewers may be ableto go back from the related content to the time marker in the videowhere it occurs.

In one aspect, the videos and content presented to a viewer may be ownedby a single content provider. Thus, as the user selects videos to play,the content provider may select other content to present to the userfrom within the content providers' data repository. Video and contentmay also be owned by third party content providers and presented as theuser selects videos to play.

Referring now to FIG. 1, illustrated therein is an example connectivitysystem 100 for implementing various aspects of the present invention.Digital device 102 is communicatively coupled to server 106 via anaccess network 104, such as the Internet. Digital device 102 may includeany suitable mobile, portable computing or communications device, suchas a cellular device, coupled to a server, such as a PC, minicomputer,mainframe computer, microcomputer, or other device having a processorand a repository for data and/or connection to a repository for data,via, for example, an access network, such as the Internet or anintranet. For example, digital device 102 may include a cellulartelephone, a navigation system, a global positioning system (GPS), acomputing device, a camera, a personal digital assistant (PDA), personalcomputers (PCs), tablets, minicomputers, mainframe computers,microcomputers, telephonic devices, or other handheld device havingwireless connection capability, among other devices. Server/computingdevice 106 may include any mobile or fixed computing device connected toa network. Sever/computing device 106 may include, but is not limitedto, a computing device, a server, a cellular telephone, a camera, a PDA,a music device, a gaming device, a navigation system, or a handhelddevice having wireless connection capability, among other devices.Furthermore, access network 104 may provide one or more types ofcommunication connections, such as any type of wireless airlink, todigital device 102 and server 106.

Digital device 102 may receive a video 14 selected by a user of digitaldevice 102 and present the selected video 14 on interface display 20.Digital device 102 may also receive related content 16 associated withthe selected video 14 from server 106 and present the related content 16concurrently with the video 14 on interface display 20. Digital device102 may include one or more input devices, including but not limited toa keyboard, a number pad, a mouse, a touch-sensitive display, anavigation key, a function key, a microphone, a voice recognitioncomponent, any other mechanism capable of receiving an input from auser, or any combination thereof. Further, digital device 102 mayinclude one or more output devices, including but not limited to adisplay, a speaker, or any other mechanism capable of presenting anoutput to a user.

In addition, digital device 102 may interface with a request component10 operable to receive a video request 12 for video 14 from a user ofdigital device 102. In an aspect, digital device 102 may becommunicatively coupled to request component 10 via, for example, accessnetwork 104. Digital device 102 may communicate the received videorequest from the user of digital device 102 to request component 10.Request component 10 may present a user with recommended videos that areavailable for viewing from a content provider. The user may select avideo to view, for example, by clicking on a title and/or an image ofthe video, using a drop down menu, or inputting text to select video,among other user interface mechanisms for inputting information.

In an aspect, upon selecting a video to view, the user may be promptedwith an option to select whether to pay a price to view a video withoutsponsor advertisements. For example, a user may pay a nominal fee, e.g.,$0.99, to view videos for a day without viewing the sponsoradvertisements or select to pay more, e.g., $4.99, to watch videos for amonth without viewing the sponsor advertisements. The user may also beprovided with an option to select to view a video for free with sponsoradvertisements. In an aspect, the user may select which sponsorsadvertisements they would like to view, e.g., the user may select thecategory of technology for which to view advertisements from technologysponsors. Thus, the user may tailor the video viewing experience byelecting not to receive sponsor advertisements and/or electing whichcategories of advertisements the user would like to view.

In another aspect, server 106 may select videos to present to a userbased upon user preferences, user demographics (e.g., age, gender),and/or user browsing behavior. For example, if the user's browsingbehavior indicates that the user accesses articles about cooking on afrequent basis, server 106 may select a cooking video to present to theuser.

Request component 10 may send the video request 12 along with theidentified video 14 to server 106. Server 106 may have a receivingcomponent 30 operable to receive the video request 12 from requestcomponent 10 and retrieve the identified video 14. For example,receiving component 30 may access a video data repository 46 with aplurality of videos 15 and retrieve the identified video 14 from thevideo data repository 46.

Request component 10 may send the retrieved video 14 to video transcriptcomponent 32 for processing. Video transcript component 32 may generatea video transcript file 34 by receiving the retrieved video 14 andconverting the audio of video 14 into text.

Video transcript component 32 may also include a conversion component 52operable to convert the audio of video 14 into text. For example,conversion component 52 may use one or more speech-to-text technologieswhen converting the audio into text. Referring now to FIG. 2, videotranscript component 32 may apply one or more extraction rules 50 tovideo 14. Extraction rules 50 may determine how meaning of the texttranscript should be determined. For example, the extraction rules mayidentify specific character names in the video, locations in the video(e.g., geographical locations, restaurants, stores, shopping centers,cafes, etc.), and social media accounts associated with the charactersin the video (e.g., hashtags, account information, etc.) and maydetermine how the identified information should be processed. Inaddition, the extraction rules may define words that may bedisambiguated by definition and/or context.

Video transcript component 32 may further include an entity extractioncomponent 54 operable to extract one or more entities 56 from the textof the video. Entity extraction component 54 may use the one or moreextraction rules or lists 50 to identify and classify specific terms inthe text transcript of the video. For example, the identified terms inthe video may be categorized into entities by using natural languageprocessing to identify key concepts in the video (e.g., vanilla cake)and may improve the entity extraction process. In addition, theextraction may disambiguate words by definition and/or context. Thus, ifthe video transcript mentions the word “buddy” the entity extraction mayuse the extraction rules to determine that the word “buddy” refers to acharacter's name in the video instead of a friend.

In addition, video transcript component 32 may generate a videotranscript markup file 58 with time markers 60 associated with the oneor more extracted entities 56. The time markers may delineate each blockof closed captioning text to appear with the video image at the timemarker corresponding to the extracted entities (e.g., when the wordsdescribing the entities are spoken). In addition, the video transcriptfile may also include metadata labels associated with the entitiesidentifying how the entity is classified. For example, the videotranscript file may indicate that New York is a location. In an aspect,the video text transcript file may be a XML or JSON file.

Referring back to FIG. 1, server 106 may also include related contentengine 36 which is operable to determine related content 16 to video 14.For example, related content engine 36 may receive the video transcriptmarkup file 58 from video transcript component 32. Related content mayinclude, but is not limited to, content that is determined to have arelationship with the video, including, but not limited to, games,activities, books, maps, social media applications, weatherapplications, shopping applications, advertisements, promotions, news,recipes, and articles, among other forms of content. Relationships maybe based upon various data sources, such as, genre data, location data,metadata entered by a content producer (e.g., title, keywords, networkinformation, accounts associated with the content, date information),downloads/streams initiated by others, downloads/streams initiated bythe user, a browsing history of a user, and social network data, amongother data and/or metadata sources By using various data sources todetermine the related content, aligned in time, for example, by usingthe video transcript markup file 58 as the master, the determinedrelated content may include a diverse group of content.

Referring now to FIG. 3, related content engine 36 may include a contentretrieval component 62 operable to retrieve related content 16 from oneor more content data repositories 48. The content data repositories maybe owned by the same content provider of the video, third party publicfeeds (e.g., social media accounts, websites, maps, etc.), and/orpartner or sponsored data repositories. In addition, content retrievalcomponent 62 may also retrieve facets 64 from the related content 16.Facets may correspond to properties of the source for the relatedcontent and may include pre-existing fields in a data repository, suchas a classification, description, or format. For example, contentretrieval component 62 may retrieve specific facet types (e.g.biography) from an article data repository using extraction rules 50.Once the entity and corresponding facets are extracted using naturallanguage processing (NLP) techniques and extraction rules 50, the systemmay retrieve related content items from relevant related contentrepositories. Thus, by using faceted classification, pre-definedextraction rules and NLP techniques, the transcribed text may be used toretrieve content from relevant related content repositories rather thanretrieving related content from all available content repositories orusing words from the transcript without additional processing.

Content retrieval component 62 may also include related content rulescomponent 66 that may be used to identify which related content 16 toretrieve and from which related content sources the related content 16should be retrieved. In an aspect, related content rules component 66may apply one or more rankings and/or quality standards to potentialrelated content. For example, related content rules component 66 mayapply a hierarchy to content providers that may supply related content16. Related content rules component 66 may instruct the contentretrieval component 62 to retrieve related content 16 from a specificcontent providers first, and if the related content 16 is not availablefrom the specific content provider, to retrieve the related content 16from a content provider with the next highest ranking. In addition,related content rules component 66 may apply a set of quality standardscriteria to determine whether or not related content 16 should beretrieved. Quality standards criteria may include, but are not limitedto, images with a specified size and/or resolution, word count limits,and/or text containing specific information (e.g., directions,preparation times, and/or ingredients), among other quality standards.For example, related content rules component 66 may only include relatedcontent 16 for retrieval if the selected related content includes imagesof a certain resolution. If the selected related content does notinclude the images of certain resolution, the related content rulescomponent 66 may determine that the selected related content does notmeet the quality standards and may instruct content retrieval component62 to select different related content 16 from the content datarepositories for video 14.

The related content retrieval rules may increase the likelihood of arelevant match and/or a higher quality match between the related contentand the selected video. In addition, the related content rules mayreduce the processing time to determine related content to the video.

Content retrieval component 62 may also include matching component 68operable to match related content 16 to the extracted entities of video14. For example, a recipe article for making a pie which has beencategorized as a recipe facet may be matched to a paragraph of text fromthe video transcript about baking a pie. Because the word pie appears inthe video transcript with the word baking, the entity extractioncomponent 54 can determine that the video transcript is talking aboutfood instead of a pie chart, for example. The matching component 68 maymatch the entities extracted from video 14 with the recipe article basedon the facet of the article and the subject matter of the article.

Related content engine 36 may also include a tagging component 70operable to manually tag related content 16 with meta-data referencingrelated content 16. For example, the meta-data may summarize andcategorize the related content 16.

In addition, related content engine 36 may include a thumbnail component72 operable to create a dynamic thumbnail from images within the relatedcontent. Dynamic thumbnails may include, for example, images createdfrom the related content (e.g., a screen grab from a video and/or animage within the source content). The images included in the dynamicthumbnails may be a thumbnail sized image created from a related contentarticle's original sized image and/or from a screen grab of a video. Inaddition, dynamic thumbnails may also include excerpts from the relatedcontent that provide an overview of the related content. In an aspect,the dynamic thumbnails may be included in menu lists or preview bubbleto allow users to preview the related content before selecting to viewthe related content.

Related content engine 36 may further include a content feed component74 operable to generate a content feed 38 including video 14, relatedcontent 16, thumbnails 76, and video transcript markup file 58, asillustrated, for example in FIG. 4.

Referring back to FIG. 1, server 106 may additionally include a contentdelivery manager component 40 operable to interface with related contentengine 36 and deliver content feed 42 to digital device 102.

Digital device 102 may include display component 17 operable to receivecontent feed 42 from content delivery manager component 40. Displaycomponent 17 may also include interface display 20 operable to presentcontent feed 42 to a user of digital device 102. Example interfacedisplays 20 are illustrated in FIGS. 9A-9M).

Digital device 102 may also include a navigation component 22 operableto navigate between the playing video 14 and various related content 16displayed contemporaneously with video 14.

Digital device 102 may also include a sharing component 28 operable toshare an experience related to the user's interaction with interfacedisplay 20. For example, sharing component 28 may share a recording ofthe video a user viewed and the related content presented to the user.In an aspect, a browsing history data store may maintain a recording ofwhich related content the user selected to view in more detail while thevideo was playing. In an aspect, the user may use the sharing component28 to share the recorded experience, e.g., video and related content,with other individuals who may receive a recording of the same videoexperience that the user selected to view and/or interact with content.It should be noted that the browsing history data store may havemultiple recorded experiences for a user, and therefore, a user may havean option to select one or more recorded experiences to share with otherindividuals. For example, the user may share the video viewingexperience via social media accounts so that other users may follow thesame experiences, such as viewing the video and the related content thatthe user selected to view.

Sharing component 28 may further include a quote component 29 configuredto grab a quote from the playing video and/or grab a portion of theplaying video to share with other individuals. In an aspect, quotecomponent 29 may allow a user to select a portion of the transcript forthe video currently playing and share the quote from the selected text.The quote component 29 may allow the user to select a portion of thevideo transcript, for example, for the previous 10 seconds, to includewith the selected quote to put the quote in context of the video. Thus,the user may have the option to select just the quote and/or selectadditional text surrounding the quote in order to provide additionalcontext for the quote.

In addition to sharing the quote, quote component 29 may share theportions of the video that mention the quote. For example, a link to thevideo segment corresponding to the quote grabbed may be shared alongwith the quote. The link to the video segment may be queued to the pointin the video where the grabbed quote is located. Thus, when a userclicks on the video link, the user may access the location of the videowhere the quote occurred without having to watch the entire videosegment.

In an aspect, quote component 29 may also allow a user to customizewhich time segments of the playing video to grab. For example, theexperience provider may preset quotes that the content provider may wanta user to share with other individuals. In another example, the user mayselect a start time segment, e.g., 2:10, of the video playing and an endtime segment, e.g., 2:25 seconds, of the playing video. The quotecomponent 29 may grab a copy of the video between 2:10 seconds and 2:25seconds and allow the user to share the grabbed video. In an aspect, auser may grab a quote from the video transcript and/or a portion of theplaying video using the quote component 29, and share the selected quoteand/or video via the user's social media accounts.

Digital device 102 may also include a feedback component 24 operable toreceive feedback 26 from a user of digital device 102. Feedback mayinclude, for example, reviewing and/or rating content, providingcomments on content, sharing content with other users, and selectingcontent to view and/or access, among other forms of feedback the usermay provide regarding the related content.

Feedback component 24 may transmit the feedback 26 to optimizationcomponent 44 on server 106. Optimization component 44 may receivefeedback 26 from feedback component 24 and apply one or more analyticsto measure the interaction with each piece of related content 16 basedupon the feedback 26 associated with related content 16. The analyticsmay monitor specific terms, categories, timeline proximity, andfrequency of interactions with the related content to determine whichrelated content 16 produces the best engagement metrics. For example, ifrelated content associated with one character receives more user viewsthan related content associated with another character, optimizationcomponent 44 may provide a revised content feed 45 with related contentassociated with the character that receives more user views than theother character.

Additionally, the analytics may determine that particular types ofrelated content may be more frequently viewed than other types ofrelated content. For example, if the analytics determine that actorbiographies are read for longer time periods than information aboutplaces, optimization component 44 may increase the number of actorbiographies associated with the video. The analytics may also determinewhether content that is a certain time period away from the previouscontent is viewed more or less, and optimization component 44 may adjustthe component based upon the determination. For example, if the contentis spaced too closely together and the content is not viewed,optimization component 44 may increase the time period between thecontent. The analytics may further determine whether the total amount ofrelated content presented with the video is appropriate. For example,optimization component 44 increase and/or decrease the total amount ofrelated content presented with the video. Thus, the optimization processmay also smooth out the video viewing experience and update the rulesused to determine the related content.

In an aspect, the optimization process may personalize the viewingexperience for a user by changing the related content presented to theuser based upon the user's previous viewing behavior. For example, ifthe user typically selected to play games instead of viewing recipes,the related content may present more games associated with the videoinstead of recipes. Thus, the relevancy of the content presented overtime may improve as the system learns the user's preferences through,for example, the received user feedback (e.g., information provided bythe user) and/or usage behaviors of the user (e.g., content the userselects to view and/or content that the user selects not to view). Inaddition, the relevancy of content presented may be based upon contentother users select to view. For example, if a video clip is viewed by100 users and majority of the users select the same article to read, thesystem may infer that the article is related to the video based upon thebehavior of other users who viewed the same video. Therefore, the morethe user interacts with the system, the more personalized the viewingexperience may become for the user.

Referring now to FIG. 5, illustrated therein is a method 500 forcreating a video transcript in accordance with an aspect of the presentinvention. The method may include retrieving a video 502. For example,receiving component 30 (FIG. 1) on server 106 (FIG. 1) may retrieve theidentified video from a data repository of videos.

The method may also include converting speech from the retrieved videointo a text transcript 504. For example, conversion component 52 (FIG.2) may use speech to text technology to convert the audio of the spokenwords in the video into a text transcript of the video. In anotheraspect, the video text transcripts may be retrieved from a datarepository of video transcripts.

The method may also include determining if any extraction rules apply tothe retrieved video transcript 506. Extraction rules may include, butare not limited to, rules regarding the transcription of the selectedvideo. For example, extraction rules may identify specific characternames in the video, locations in the video (e.g., geographicallocations, restaurants, stores, shopping centers, cafes, etc.), andsocial media accounts associated with the characters in the video (e.g.,hashtags, account information, etc.) and determine how the identifiedinformation should be processed.

In addition, the method may include applying the determined extractionrules to the retrieved video transcript 508. In an aspect, extractionrules component 50 (FIG. 2) may determine if any extraction rules applyto the retrieved video transcript, and may apply the determinedextraction rules to the words spoken in the retrieved video transcript.For example, when the identified words are spoken, e.g., a character'sname, in the retrieved video, the extraction rules may determine thatthe character's social media account should be associated with thecharacter's name.

The method may further include extracting one or more entities from theretrieved video transcript 510. In an aspect, entity extractioncomponent 54 (FIG. 2) may identify and classify specific terms in thetext transcript of the video. For example, the identified terms in thevideo may be categorized into entities by using natural languageprocessing which may use meta-data lists to more precisely process andcategorize the identified terms into entities and may improve the entityextraction process. For example, if the text transcript referenced acharacter's bakery in the video, entity extraction component may extractthe character's bakery from the text transcript and classify the bakeryas a “place” entity. In addition, by using natural language processing,the character's bakery may also be identified as the place where thecharacter works. Thus, using natural language processing and extractionrules may improve the matching process between the transcribed text andthe entities extracted from the transcribed text.

In addition, the method may additionally include associating the one ormore entities with time markers in the video 512. The time markers maydelineate each block of closed captioning text to appear with the videoimage at the time the corresponding to the extracted entities (e.g.,when the words describing the entities are spoken).

The method may also include generating a video transcript file with thetime markers associated with the one or more entities 514. For example,video transcript component 32 may generate a transcript file that maycontain a text transcript of the video and time markers associated withthe one or more entities. This video transcript file may be used tomatch entities in the video with related content aligned to the timemarkers in the video. In an aspect, the video transcript file may be aXML or JSON file.

Referring now to FIG. 6, illustrated therein is a method 600 fordetermining related content to the video in accordance with an aspect ofthe present invention. Related content may include content that isdetermined to have a relationship with the video. Relationships may bebased upon various data sources, such as, genre data, location data,metadata inserted by a content producer, downloads performed by others,downloads performed by the user, the user's browsing history, and socialnetwork data, among other data sources. Related content may include, butis not limited to, games, activities, books, social media applications,weather applications, shopping applications, advertisements, promotions,news, recipes, and articles, among other forms of content.

The method may include receiving a video transcript file with timemarkers associated with one or more entities of the video 602. Forexample, related content engine 36 may receive the video transcript filethat may contain a text transcript of the video and time markersassociated with one or more entities from the video.

The method may further include selecting related content for theentities based upon applied related content rules 604 and time-aligningthe related content selections with the video. Related content engine 36(FIG. 3) may use, for example, related content rules component 66 fordetermining relationships between the content and the extracted entitiesfrom the video. In an aspect, related content rules component 66 may usethe extracted entities from the video as a dynamic search query fordynamically determining which content in a data repository may berelated to the video. In addition, the related content rules component66 may use elements of the video in combination with other data sources(e.g., location, genre data, metadata, etc.) for dynamically determiningrelated content for the entities. By using related content rules, theprocessing time for determining which related content to use may bereduced and the relevance of the related content that is retrieved maybe increased.

For example, searching for related content may be performed based uponkeywords from the text transcript. When keywords from the texttranscript match the content, the content may be selected as beingrelated to the video. For example, if the video transcript includes“Alaskan King Crabs” the server may select an article on fishinginjuries related to Alaskan King Crabs as being related to the video.

It should be noted that a variety of searching methods may be used incombination with and/or in addition to keyword searching. For example,the searching may infer the meaning of content by using techniques, suchas Bayesian pattern matching techniques and selecting content based uponthe inferred meaning. In addition, the searching may disambiguate thesearch queries terms and remove content that may not be relevant to thesearch query

In addition, the method may include extracting facets from the selectedrelated content 606. For example, related content engine 36 may extractone or more facets from the related content data repositories. In anaspect, related content rules component 66 may apply one or more contentrules that define relationships between the entities and the facets andmay determine related content items for the entities based upon thedefined relationships between the entities and the facets. In addition,content retrieval component 62 may retrieve the determined relatedcontent items from one or more content data repositories.

Optionally, the method may include applying one or more tags to manuallyinsert meta-data to the related content 608. For example, taggingcomponent 70 (FIG. 3) may tag the related content with meta-datadescribing the related content. For example, the meta-data may summarizeand categorize the related content.

The method may further include creating one or more dynamic thumbnailsrepresenting the related content 610. Dynamic thumbnails may include,for example, images created from the related content (e.g., a screengrab from a video and/or an image within the source content). Forexample, the images included in the dynamic thumbnails may be athumbnail sized image created from a related content article's originalsized image and/or from a screen grab of a video. In addition, dynamicthumbnails may also include excerpts from the related content thatprovide an overview of the related content. In an aspect, the dynamicthumbnails may be included in menu lists to allow users to preview therelated content before selecting to view the related content.

The method may also include generating a content feed for presentationon a digital device 612. For example, the content feed may include thetranscript file, the dynamic thumbnails and the related content.

The method may optionally include revising the content feed 614. In anaspect, the system may revise the content feed based upon feedbackreceived from users of the system. For example, the system may receivefeedback from users viewing the videos and related content. The receivedfeedback may signal whether the user is interested and/or not interestedin the presented content. For example, a user of the digital device mayreview and/or rate the content, provide comments on the content, sharethe content with other users, select content to view and/or access,select not to view and/or access content, and purchase a promotion,among other forms of feedback. The system may apply one or moreanalytics to measure the interaction with each piece of related contentbased upon the received feedback associated with the related content andmay determine which related content produces the best engagementmetrics. The system may revise the content feed based upon the receivedfeedback.

In an aspect, the content feed may be reviewed and/or revised by acontent producer. The content producer may view the video with theclosed captioning and related content to approve, modify, add or deleterelated content matches. The content producer can manually apply similarcontent rules to optimize the related content associated with the video.The content producer may also apply other resources, such as aneditorial calendar or current events, to ensure the related contentcorresponds to events that are currently taking place.

Referring now to FIGS. 7A and 7B, illustrated therein are example methodflows for entity extractions and facet extractions in accordance with anaspect of the present invention. FIG. 7A illustrates an example methodthat applies an entity extraction rule corresponding to determiningwhether the word and/or phrase from the video correspond to a type offood. In addition, the method includes an example facet extraction froma content data repository for content associated with food, such asrecipe articles. The method also includes matching the food word to arecipe about the food word.

FIG. 7B illustrates an example method that applies an entity extractionrule corresponding to determining whether the word and/or phrase fromthe video correspond to a character name. The method also includes abiography facet extraction from the content data repository for contentassociated with a biography. The method further illustrates matching theextracted facets, e.g., biographies from the content data repository,with the extracted entity, e.g., character name, to match the charactername with a biography of the character.

Referring now to FIG. 8, illustrated therein is an example method flowfor determining related content for the video in accordance with anaspect of the present invention. The method may include determiningwhether the word in the video transcript is a type of food, a charactername and/or a place. The method may further include extracting relatedcontent associated with the word from the video transcript anddetermining whether the extracted related content passes a qualitystandard. If the related content passes the quality standard, the methodmay further include adding the related content to the transcript file.If the related content does not pass the quality standard, the methodmay further include extracting a different type of related content forthe word in the video transcript and determining whether the newlyextracted related content passes the quality standard. Thus, it shouldbe noted that method may repeat until the extracted content passes therelated quality standards.

Referring now to FIGS. 9A-9M, illustrated therein are example graphicaluser interface (GUI) screens illustrating exemplary implementations ofaspects of the present invention. FIG. 9A illustrates an exampleinterface 900 in accordance with an aspect of the present invention.Interface 900 may include a video player interface 902 operable to playa selected video. Interface 900 may also include one or more relatedcontent tabs 904 corresponding to one or more related content. Forexample, each related content tabs 904 may represent a specific categoryof content, such as a recipe, a photo gallery, sponsored content, agame, an article, video clips, social media feeds, maps, and locations,among other categories of content. Thus, when a content tab is selectedby a user, the related content associated with the selected tab may bepresented. Interface 900 may also include a related content interface906 where related content may be dynamically presented concurrentlywhile the video is playing in video player interface 902.

Interface 900 may also include an interactive timeline 914 withcorresponding timeline icons 912. The interactive timeline icons mayinclude graphical representations corresponding to the type and/orcategory of the related content associated with the interactive timelineicons. Timeline icons 912 may also control the presence of relatedcontent icons on interactive timeline 914. In addition, when thetimeline icons 912 are selected, the corresponding related content maybe presented in the related content interface 906.

Interface 900 may further include a closed captioning button 910. FIG.9B illustrates closed captioning text presented on interface 900 alongwith the video playing in video player interface 902.

FIG. 9C illustrates a preview bubble 918 with a thumbnail image ofrelated content that corresponds to timeline icons 912 (FIG. 9A). Thepreview bubble 918 alerts the user to related content that may bedisplayed and/or already displayed. Interface 900 may further include acontent stream control button 922 that allows a user to resume and/orpause the content stream. The related content may automatically load inthe related content interface 906 when events in the playing videocorrespond to the related content interface 906. In addition, thepreview bubble 918 may also be synchronized and present the relatedcontent when the events in the playing video correspond to the relatedcontent. Thus, when the playing video reaches a related content item,the preview bubble for the related content may appear over theinteractive timeline and/or over the video. By turning the contentstream control button 922 to “off”, the related content does not loaduntil either the user selects “on” or “resume”, or clicks on a relatedcontent tab or icon on the interactive timeline. Interface 900 may alsoinclude a list all available content button 920. By selecting the listall available content button 920, interface 900 may present a list ofall the available content associated with the playing video capable ofbeing presented in the related content interface 906. The user mayselect a variety of related content from the list and have the selectedrelated content be presented in the related content interface 906 and/orthe video move to the time marker associated with the selected relatedcontent.

FIG. 9D illustrates a related content panel 928 displayed on interface900. Interface 900 may include a related content stream setting 926 thatmay control the automatic load of the related content. For example, therelated content may automatically load when synchronized with thepreview bubble events overlaying the interactive video timeline or therelated content may not load until the user selects “on” or “resume.” Inaddition, interface 900 may include interactive timeline cue icons 924that control the presence of related content icons on the interactivetimeline 914.

FIG. 9E illustrates an ability to hide related content interface 906 byselecting the button 930. By hiding the related content interface 906,the video player interface 902 may expand and cover most of interface900. Interface 900 may also include a full expansion button 932 that mayfully expand the related content interface 906. When a user selects thefull expansion button 932, the video may pause and the related contentinterface may cover most of interface 900. Interface 900 may alsoprovide a user an option to access one or more social media accounts byselecting social media button 934. In addition, interface 900 mayprovide real-time social media content 938, as illustrated in FIG. 9F.It should be noted that the user may access and/or interface with theone or more social media accounts while simultaneously viewing the videoplaying in video play module.

FIG. 9G illustrates an example extended interview and biography 940 of acharacter in the video playing in the video player interface 902. Theuser may pause the video while reviewing the extended interview andbiography 940 of the character before resuming the video.

As illustrated in FIG. 9H, interface 900 may allow a user to grab aquote from the video playing in video player interface 902. The quotemay be an editorial quote 942 where the content provider of the videopre-selects quotes from the video. In addition, a user may be able tohighlight a section of the transcript to create a quote from the videoto share with other individuals, as illustrated in 946 of FIG. 9I. Byproviding the text transcripts, users are able to share the exact quotefrom the video. Interface 900 may also allow a user to share theselected quote via one or more social media accounts 944. For example,the user may share the selected quote by importing the quote into asocial media message (e.g., a Tweet or a Facebook message).

In addition, as illustrated in FIG. 9I, interface 900 may allow a userto share the selected quote along with a link to the specific moment inthe video associated with the quote 945. A user may be able to select toemail or embed a link to the beginning of a video to share an entirevideo with another user. The user may also select to share the video andrelated content elements viewed during the video session. Thus, anotherindividual may be able to have the same video experience as the usersharing the video and related content.

FIG. 9J illustrates an example map 948 related to a place mentioned inthe video presented in the related content interface 906.

FIG. 9K illustrates an example promotion 950, such as an offer ordiscount for the related content presented in related content interface906. In addition to promotions, sponsors may advertise in relatedcontent interface 906 to drive user awareness of the various productsassociated with the presented related content. For example, if the videoplaying is related to catching lobsters, interface 900 may presentpromotions 950 relating to seafood restaurants. In an aspect, the servermay receive location information from the digital device via a locationcomponent, such as global positioning system (GPS), and use the receivedlocation information in providing targeted advertising to the user forplaces near the received digital device location. Another example mayinclude presenting a coupon for purchasing canned tomatoes when thevideo being played is a video demonstrating cooking Italian food. Thus,the advertisements and/or promotions presented may be tailored and/orchanged based upon the video that is currently being played in videoplayer interface 902. Content providers may have content presented withintegrated advertisements and/or promotions based upon the videosselected by users. In an aspect, a third party may broker relationshipsamong content providers and advertisers.

FIG. 9L illustrates an example interface 900 where a user may play agame in the related content interface 906 while the video continues toplay in the video player interface 902. It should be noted that thegames and other activities, such as such as puzzles and quizzes may bedynamically displayed to the user based on their determined relevance tothe selected video.

FIG. 9M illustrates an example interface 900 where a related video isavailable for viewing in related content interface 906 whilesimultaneously watching the main video in the video player interface902.

As used in this application, the terms “component,” “module,” “system”and the like are intended to include a computer-related entity, such asbut not limited to hardware, firmware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to being, a process running on a processor, aprocessor, an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a computing device and the computing device can be a component. Oneor more components can reside within a process and/or thread ofexecution and a component may be localized on one computer and/ordistributed between two or more computers. In addition, these componentscan execute from various computer readable media having various datastructures stored thereon. The components may communicate by way oflocal and/or remote processes such as in accordance with a signal havingone or more data packets, such as data from one component interactingwith another component in a local system, distributed system, and/oracross a network such as the Internet with other systems by way of thesignal.

Aspects of the present invention may be implemented using hardware,software, or a combination thereof and may be implemented in one or morecomputer systems or other processing systems. In an aspect of thepresent invention, features are directed toward one or more computersystems capable of carrying out the functionality described herein. Anexample of such a computer system 1000 is shown in FIG. 10.

Computer system 1000 includes one or more processors, such as processor1004. The processor 1004 is connected to a communication infrastructure1006 (e.g., a communications bus, cross-over bar, or network). Varioussoftware aspects are described in terms of this exemplary computersystem. After reading this description, it will become apparent to aperson skilled in the relevant art(s) how to implement aspects of theinvention using other computer systems and/or architectures.

Computer system 1000 can include a display interface 1002 that forwardsgraphics, text, and other data from the communication infrastructure1006 (or from a frame buffer not shown) for display on a display unit1030. Computer system 1000 also includes a main memory 1008, preferablyrandom access memory (RAM), and may also include a secondary memory1010. The secondary memory 1010 may include, for example, a hard diskdrive 1012 and/or a removable storage drive 1014, representing a floppydisk drive, a magnetic tape drive, an optical disk drive, a universalserial bus (USB) flash drive, etc. The removable storage drive 1014reads from and/or writes to a removable storage unit 1018 in awell-known manner. Removable storage unit 1018 represents a floppy disk,magnetic tape, optical disk, USB flash drive etc., which is read by andwritten to removable storage drive 1014. As will be appreciated, theremovable storage unit 1018 includes a computer usable storage mediumhaving stored therein computer software and/or data.

Alternative aspects of the present invention may include secondarymemory 1010 and may include other similar devices for allowing computerprograms or other instructions to be loaded into computer system 1000.Such devices may include, for example, a removable storage unit 1022 andan interface 1020. Examples of such may include a program cartridge andcartridge interface (such as that found in video game devices), aremovable memory chip (such as an erasable programmable read only memory(EPROM), or programmable read only memory (PROM)) and associated socket,and other removable storage units 1022 and interfaces 1020, which allowsoftware and data to be transferred from the removable storage unit 1022to computer system 1000.

Computer system 1000 may also include a communications interface 1024.Communications interface 1024 allows software and data to be transferredbetween computer system 1000 and external devices. Examples ofcommunications interface 1024 may include a modem, a network interface(such as an Ethernet card), a communications port, a Personal ComputerMemory Card International Association (PCMCIA) slot and card, etc.Software and data transferred via communications interface 1024 are inthe form of signals 1028, which may be electronic, electromagnetic,optical or other signals capable of being received by communicationsinterface 1024. These signals 1028 are provided to communicationsinterface 1024 via a communications path (e.g., channel) 1026. This path1026 carries signals 1028 and may be implemented using wire or cable,fiber optics, a telephone line, a cellular link, a radio frequency (RF)link and/or other communications channels. In this document, the terms“computer program medium” and “computer usable medium” are used to refergenerally to media such as a removable storage drive 1080, a hard diskinstalled in hard disk drive 1070, and signals 1028. These computerprogram products provide software to the computer system 1000. Theinvention is directed to such computer program products.

Computer programs (also referred to as computer control logic) arestored in main memory 1008 and/or secondary memory 1010. Computerprograms may also be received via communications interface 1024. Suchcomputer programs, when executed, enable the computer system 1000 toperform the features in accordance with aspects of the presentinvention, as discussed herein. In particular, the computer programs,when executed, enable the processor 1004 to perform the features inaccordance with aspects of the present invention. Accordingly, suchcomputer programs represent controllers of the computer system 1000.

In an aspect of the present invention where the invention is implementedusing software, the software may be stored in a computer program productand loaded into computer system 1000 using removable storage drive 1014,hard drive 1012, or communications interface 1020. The control logic(software), when executed by the processor 1004, causes the processor1004 to perform the functions described herein. In another aspect of thepresent invention, the system is implemented primarily in hardwareusing, for example, hardware components, such as application specificintegrated circuits (ASICs). Implementation of the hardware statemachine so as to perform the functions described herein will be apparentto persons skilled in the relevant art(s).

In yet another aspect of the present invention, the invention isimplemented using a combination of both hardware and software.

FIG. 11 shows a communication system 1100 usable in accordance withaspects of the present invention. The communication system 1100 includesone or more accessors 1160, 1162 (also referred to interchangeablyherein as one or more “users”) and one or more terminals 1142, 1166. Inone aspect of the present invention, data for use is, for example, inputand/or accessed by accessors 1160, 1164 via terminals 1142, 1166, suchas personal computers (PCs), minicomputers, mainframe computers,microcomputers, telephonic devices, or wireless devices, such aspersonal digital assistants (“PDAs”) or a hand-held wireless devicescoupled to a server 1143, such as a PC, minicomputer, mainframecomputer, microcomputer, or other device having a processor and arepository for data and/or connection to a repository for data, via, forexample, a network 1144, such as the Internet or an intranet, andcouplings 1145, 1146, 1164. The couplings 1145, 1146, 1164 include, forexample, wired, wireless, or fiberoptic links. In another aspect of thepresent invention, the method and system of the present inventionoperate in a stand-alone environment, such as on a single terminal.

While aspects of the present invention have been described in connectionwith examples thereof, it will be understood by those skilled in the artthat variations and modifications of the aspects of the presentinvention described above may be made without departing from the scopehereof. Other aspects will be apparent to those skilled in the art froma consideration of the specification or from a practice in accordancewith aspects of the invention disclosed herein.

What is claimed is:
 1. A method for enhancing a digital contentexperience, the method comprising: receiving a video transcript filewith a text transcript of a video; identifying one or more keywords inthe text transcript of the video; extracting, via interpretation of theone or more keywords, and using time markers corresponding to the one ormore keywords, one or more key concepts corresponding to the one or moretime markers in the video; retrieving one or more related content itemscorresponding to the extracted key concepts, wherein the related contentitems are retrieved from sources external to the video and the videotranscript file; applying one or more content rules to definerelationships between the extracted key concepts and facets of potentialrelated content items to determine related content items for theextracted key concepts, wherein the one or more content rules apply oneof a ranking standard and a quality standard to the potential relatedcontent items; determining whether the potential related content itemspasses the quality standard; retrieving a different type of relatedcontent items corresponding to the extracted key concepts as a result ofdetermining that the potential related content items did not pass thequality standard; determining related content items corresponding to theextracted key concepts as a result of determining that the potentialrelated content items passed the quality standard; associating thedetermined related content items to the time markers corresponding tothe extracted key concepts; generating a content feed comprising therelated content items and the video transcript file; and transmittingthe content feed and the video for contemporaneous presentation on adigital device.
 2. The method of claim 1, wherein at least one keyconcept is extracted via an interpretation of two or more keywords inthe text transcript of the video.
 3. The method of claim 1, furthercomprising: revising the related content items in the content feed basedupon received feedback associated with the related content items.
 4. Anapparatus for enhancing a digital content experience, the apparatuscomprising at least one computer processor configured to: receive avideo transcript file with a text transcript of a video; identify aplurality of specific terms in the text transcript of the video;classify each of the plurality of specific terms into one or moreentities using one or more extraction rules; associate each of theentities with a time marker in the video transcript file correspondingto one or more of the specific terms; retrieve related content itemsexternal to the video and the video transcript file for the entities;apply one or more content rules to define relationships between entitiesand facets of potential related content items to determine relatedcontent items for the entities, wherein the one or more content rulesapply one of a ranking standard and a quality standard to the potentialrelated content items; determine whether the potential related contentitems passes the quality standard; retrieve a different type of relatedcontent items corresponding to entities as a result of determining thatthe potential related content items did not pass the quality standard;determining related content items corresponding to entities as a resultof determining that the potential related content items passed thequality standard; associate the determined related content items to thetime markers associated with the entities; generate a content feed withthe related content items and the video transcript file; and transmitthe content feed and the video for contemporaneous presentation on adigital device.
 5. The apparatus of claim 4, wherein at least one of theentities corresponds to a combination of two specific terms in the texttranscript of the video.
 6. The apparatus of claim 4, furthercomprising: an optimization component operable to revise the relatedcontent items in the content feed based upon received feedbackassociated with the related content items.
 7. A system for enhancing adigital content experience, the system comprising at least one computerprocessor, the at least one computer processor configured to executeinstructions for: receiving, at the processor, a video transcript filecomprising a text transcript of a video and entities corresponding totime markers in the video transcript file, wherein the entities arecreated by identifying and classifying specific words in the texttranscript of the video; retrieving, using the processor, relatedcontent items for the entities in the video transcript file, wherein therelated content items are from sources external to the video and thevideo transcript file; applying, using the processor, one or morecontent rules to define relationships between entities and facets ofpotential related content items to determine related content items forentities, wherein the one or more content rules apply one of a rankingstandard and a quality standard to the potential related content item;determining, using the processor, whether the potential related contentitems passes the quality standard; retrieving, using the processor, adifferent type of related content items corresponding to entities as aresult of determining that the potential related content items did notpass the quality standard; determining related content itemscorresponding to entities as a result of determining that the potentialrelated content items passed the quality standard; associating, usingthe processor, the determined related content items to the time markersfor the entities; and generating, using the processor, a content feedwith the related content items and the video transcript file;transmitting, using the processor, the content feed and the video forcontemporaneous presentation on a digital device.